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DETAILED ACTION 
Response to Appeal Brief 
1. In view of the appeal brief filed on February 24, 2004, PROSECUTION IS HEREBY 
REOPENED. New grounds of rejection are set forth below. 

To avoid abandonment of the application, appellant must exercise one of the following 
two options: 

(1) file a reply under 37 CFR 1.111 (if this Office action is non-final) or a reply under 37 
CFR 1.113 (if this Office action is final); or, 

(2) request reinstatement of the appeal. 

If reinstatement of the appeal is requested, such request must be accompanied by a 
supplemental appeal brief, but no new amendments, affidavits (37 CFR 1.130, 1.131 or 1.132) or 
other evidence are permitted. See 37 CFR 1.193(b)(2). 



Drawings 

2. The drawings are objected to under 37 CFR 1.83(a). The drawings must show every 
feature of the invention specified in the claims. Therefore, the features in claims 2-22, either by 
program diagram and\or drawings, for example the data store in claim 8 and the reading station 
in claim 21 must be shown or the feature(s) canceled from the claim(s). No new matter should 
be entered. 

Corrected drawing sheets are required in reply to the Office action to avoid 
abandonment of the application. Any amended replacement drawing sheet should include all of 
the figures appearing on the immediate prior version of the sheet, even if only one figure is being 
amended. The figure or figure number of an amended drawing should not be labeled as 
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"amended." If a drawing figure is to be canceled, the appropriate figure must be removed from 
the replacement sheet, and where necessary, the remaining figures must be renumbered and. 
appropriate changes made to the brief description of the several views of the drawings for 
consistency. Additional replacement sheets may be necessary to show the renumbering of the 
remaining figures. The replacement sheet(s) should be labeled "Replacement Sheet" in the page 
header (as per 37 CFR 1.84(c)) so as not to obstruct any portion of the drawing figures. If the 
changes are not accepted by the examiner, the applicant will be notified and informed of any 
required corrective action in the next Office action. The objection to the drawings will not be 
held in abeyance. 



Specification 

3. The attempt to incorporate subject matter into this application by reference to priority 
applicant 09/571422 is improper because the specification does not refer to any application to be 
considered as incorporated by reference. The examiner notes that since the claims are not 
supported by the parent application the effective filling date of this application is November 8, 
2000. 

4. The amendment filed March 20, 2003 is objected to under 35 U.S.C. 132 because it 
introduces new matter into the disclosure. 35 U.S.C. 132 states that no amendment shall 
introduce new matter into the disclosure of the invention. The added material, which is not 
supported by the original disclosure, is as follows: the gesture-decoding module on pages 2-6 of 
the amendment of paper number 5. 

Applicant is required to cancel the new matter in the reply to this Office Action. 
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5. The specification is objected to as failing to provide proper antecedent basis for the 
claimed subject matter. See 37 CFR 1.75(d)(1) and MPEP § 608.01(o). Correction of the 
following is required: The specification fails to disclose the reading station as recited in claims 2- 
22. 



Claim Rejections - 35 USCS112 

6. The following is a quotation of the first paragraph of 35 U.S.C. 112: 

The specification shall contain a written description of the invention, and of the manner and process of making 
and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it 
pertains, or with which it is most nearly connected, to make and use the same and shall set forth the best mode 
contemplated by the inventor of carrying out his invention. 

7. Claims 4-7 and 1M9 are rejected under 35 U.S.C. 112, first paragraph, as containing 
subject matter which was not described in the specification in such a way as to enable one 
skilled in the art to which it pertains, or with which it is most nearly connected, to make and/or 
use the invention. 

The specification fails to describe how speech is assembled from component phonemes 
or common words previously recorded by a person (claim 4), the component phonemes or 
common words are stored locally and correspond to a child, a child's family member, or a 
celebrity voice (claims 5 and 6), retrieving artwork from a data store, and printing the artwork 
for coloring by a child (claim 7), the speech incorporates both text from the book and substitute 
words (claim 11), retrieving the substitute words from a local store (Claim 12), soliciting words 
from a child, recording the child's words (claim 13), and using the recorded words as the 
substitute words, and the substitute words customize the book text to a particular child or 
locale (claim 14). Also, claims 15-19 fail to describe in the specification how one skilled in the art 
would sense a gesture from an image data to cause different events or actions to take place. The 
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examiner requests that applicant point in the original specification the claimed limitations 
noted above. Applicant is reminded that no new matter may be entered. 

8. The following is a quotation of the second paragraph of 35 U.S.C 112: 

The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the 
subject matter which the applicant regards as his invention. 

9. Claims 15-19 are rejected under 35 U.S.C. 112, second paragraph, as being indefinite for 
failing to particularly point out and distinctly claim the subject matter which applicant regards 
as the invention. 

In claim 15, the phrase, "sensing a gesture from the image data" is vague and indefinite 
because the "sensing a gesture from the image data" cannot be determined. How does one sense 
a gesture? 

In claiml6, the phrase, "one gesture causes a page from the book to be read-aloud again" 
is vague and indefinite because the one gesture causing a page from the book to be read-aloud 
cannot be determined. How can one gesture cause a page from the book to read-aloud again? 

In claim 17, the phrase, "one gesture controls volume of audio" is vague and indefinite 
because the one gesture controls volume of audio cannot be determined. How does one gesture 
control volume? 

In claim 18, the phrase, "one gesture causes text from the book to be read aloud at a faster 
speed" is vague and indefinite because the one gesture causing text from the book to be read 
aloud at a faster speed cannot be determined. How can one gesture cause a text from the book 
to be read aloud at a faster speed? 
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In claim 19, the phrase, "one gesture causes text read-aloud from the book to be read 
using a different voice is vague and indefinite because the one gesture causing a text to be read 
aloud cannot be determined. How can one gesture cause a text to be read aloud? 

Claim Rejections - 35 USCS102 

10. The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the 
basis for the rejections under this section made in this Office action: 

A person shall be entitled to a patent unless - 

(e) the invention was described in a patent granted on an application for patent by another filed in the United 
States before the invention thereof by the applicant for patent, or on an international application by another 
who has fulfilled the requirements of paragraphs (1), (2), and (4) of section 371(c) of this title before the 
invention thereof by the applicant for patent. 

The changes made to 35 U.S.C. 102(e) by the American Inventors Protection Act of 1999 
(AIPA) and the Intellectual Property and High Technology Technical Amendments Act of 2002 
do not apply when the reference is a U.S. patent resulting directly or indirectly from an 
international application filed before November 29, 2000. Therefore, the prior art date of the 
reference is determined under 35 U.S.C. 102(e) prior to the amendment by the AIPA (pre-AIPA 
35 U.S.C. 102(e)). 

11. Claim 2 is rejected under 35 U.S.C. 102(e) as being anticipated by Rhoads x 214. 

12. Rhoads discloses the following: 

Sensing a page or cover of a book with an image sensor, the sensor having 
a two-dimensional array (CCD) of optical sensing elements (column 4, lines 35- 
39), decoding a digital watermark from image data produced by the image sensor 
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and triggering an action associated with the page or cover (column 24, line 63 - 

column 25, line 4) as recited in claim 2. 
The applied reference has a common assignee with the instant application. Based upon 
the earlier effective U.S. filing date of the reference, it constitutes prior art under 35 U.S.C. 
102(e). This rejection under 35 U.S.C. 102(e) might be overcome either by a showing under 37 
CFR 1.132 that any invention disclosed but not claimed in the reference was derived from the 
inventor of this application and is thus not the invention "by another," or by an appropriate 
showing under 37 CFR 1.131. 

Claim Rejections - 35 USCS103 

13. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all obviousness 
rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in 
section 102 of this title, if the differences between the subject matter sought to be patented and the prior art are 
such that the subject matter as a whole would have been obvious at the time the invention was made to a person 
having ordinary skill in the art to which said subject matter pertains. Patentability shall not be negatived by the 
manner in which the invention was made. 

14. Claim 1 is rejected under 35 U.S.C. 103(a) as being unpatentable over Okamoto ^924 in 
view of Price ^519 and Rhoads ^214. 

15. Okamoto discloses the following: 

A stuffed toy (100) having two eyes, an image sensor having a two- 
dimensional array (or CCD), a speaker (22), and a processor coupled to the image 
sensor (CCD) as recited in claim 1. 
Okamoto does not expressly disclose the following: 

The sensing elements being positioned to view out of at least one of the 
eyes as recited in claim 1. 
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Price teaches the following: 

A doll having two eyes and a sensing element positioned to view out of at 
least one eye. By having a stuffed toy with the sensing element position to view 
out of at least one eye, one of ordinary skill in the art would provide a toy that 
exhibits a personality by responding in a different manner to different persons. 
Okamoto and Price do not expressly disclose the following: 

A steganographic watermark detector for sensing watermark data on an 
object and triggering an action in response as recited in claim L 
Rhoads teaches the following: 

A steganographic watermark detector for sensing watermark data on an 
object and triggering an action in response (abstract) as recited in claim 1. By 
sensing watermark data on an object, one of ordinary skill in the art would 
provide a system by which users can interact with computer-based devices. 
Therefore, it would have been obvious to one of ordinary skill in the art at the time of the 
invention was made to modify Okamoto to include a doll having two eyes and a sensing element 
positioned to view out of at least one eye as taught by Price and to further include a 
steganographic watermark detector for sensing watermark data on an object and triggering an 
action in response as taught by Rhoads to provide a system by which users can interact with 
computer-based devices. 

16. Claims 2 and 9 are rejected under 35 U.S.C. 103(a) as being unpatentable over Reber x 261 
in view of Rhoads M36. 

17. Reber discloses the following: 
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Sensing a page or cover of a book with an image sensor, the image sensor 
having a two-dimensional array of optical sensing elements (column 4, lines 2-5), 
and triggering an action (internet) associated with page or cover (column 5, lines 
21-36) as recited in claim 2. 

The action is linking to an Internet web site related to the book or its 
subject matter (column 5, lines 21-36) as recited in claim 9. 
Reber does not expressly disclose the following: 

Decoding a digital watermark from image data produced by the image 
sensor as recited in claim 2. 
Rhoads teaches the following: 

Decoding a digital watermark from image data produced by the image 
sensor as recited in claim 2. By decoding a digital watermark from image data 
produced by the image sensor, one of ordinary skill in the art would provide a 
multimedia enhanced reading experience. 

Therefore, it would have been obvious to one of ordinary skill in the art at 
the time of the invention was made to modify Reber to include decoding a digital 
watermark from image data produced by the image sensor as taught by Rhoads to 
provide a multimedia enhanced reading experience. 

18. Claim 3 is rejected under 35 U.S.C. 103(a) as being unpatentable over Reber "261 in view 
of Rhoads "436 as applied to claim 2 above, and further in view of Lemelson "656. 

19. Reber in view of Rhoads disclose the claimed invention as discussed above except for the 
following: 
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The associated action is speech-reciting text from a book as recited in 
claim 3. 

Lemelson teaches the following: 

The associated action is speech-reciting text from a book (summary) as 
recited in claim 3. By having speech-reciting text from a book, one of ordinary 
skill in the art would provide a child or handicapped person to derive intelligible 
information from a book. 

Therefore, it would have been obvious to one of ordinary skill in the art at 
the time of the invention was made to modify Reber to further include a speech- 
reciting text from a book as taught by Lemelson to provide a child or 
handicapped person to derive intelligible information from a book. 

20. Claims 4-6 are rejected under 35 U.S.C. 103(a) as being unpatentable over Reber ^261 in 
view of Rhoads H36 and Lemelson ^656 as applied to claims 2-3 above, and further in view of 
Linebarger ^485. 

21. Reber in view of Rhoads and Lemelson disclose the claimed invention as discussed above 
except for the following: 

The speech being assembled from component phonemes or common 
words previously recorded by a person as recited in claim 4. 

The component phonemes or common words are stored locally and 
correspond to a child or child's family member as recited in claim 5. 

The component phonemes or common words are stored remotely, and 
correspond to a celebrity voice as recited in claim 6. 
Linebarger teaches the following: 
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The speech being assembled from component phonemes or common 
words (58) previously recorded by a person (column 3, line 34 - column 4, line 
16) as recited in claim 4. 

The component phonemes or common words are stored locally (memory 
12), remotely (24), and correspond to a child or child's family member, in which 
the examiner interprets the common words stored are capable of being from a 
child or family member as recited in claim 5. By having assembling phonemes or 
common words previously recorded or stored, one of ordinary skill in the art 
would provide a communication system that is easy and simple to use. 

Therefore, it would have been obvious to one of ordinary skill in the art at 
the time of the invention was made to modify Reber to further include speech 
being assembled from component phonemes or common words previously 
recorded by a person and the component phonemes or common words are stored 
locally and correspond to a child or child's family member as taught by 
Linebarger to provide a communication system that is easy and simple to use. 
At the time the invention was made, it would have been an obvious design choice to a 
person of ordinary skill in the art to provide different voices because Applicant has not disclosed 
that phonemes or common words correspond to a celebrities voice as recited in claim 6 provides 
an advantage or solves a stated problem. On of ordinary skill in the art, furthermore, would have 
expected Applicant's invention to perform equally well with any voice recorded as taught by 
Linebarger because they provide the same function of providing a simple and easy 
communication system. 
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22. Claims 1144 are rejected under 35 U.S.C. 103(a) as being unpatentable over Reber ^261 in 
view of Rhoads x 436 as applied to claim 2 above, and further in view of La Tour X)70 and 
Linebarger x 485. 

23. Reber in view of Rhoads disclose the claimed invention as discussed above except for the 
following: 

The associated action is speech and the speech incorporates both text 
from the book and substitute words as recited in claim 11. 

Retrieving the substitute words from a local store as recited in claim 12. 

Soliciting words from a child, recording the child's words, and using the 
recorded word as the substitute words as recited in claim 13. 

The substitute words customize the book text to a particular child or 
locale as recited in claim 14. 
La Tour teaches the following: 

The associated action is speech and the speech incorporates both text 
from the book and substitute words, in which the examiner interprets the 
recording of parts of the text (word) being equivalent and capable of 
incorporating text from the book and substitute words (summary) as recited in 
claim 11. 

Soliciting words from a child, recording the child's words, and using the 
recorded word as the substitute words, in which the examiner interprets the 
recording of parts of the text (word) being equivalent and capable of 
incorporating text from the book and substitute words (summary) as recited in 
claim 13. 
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The substitute words customize the book text to a particular child or 
locale, in which the examiner interprets the multipart drama to be an equivalent 
to the customizing of the textbook (summary) as recited in claim 14. 
Linebarger teaches the following: 

Retrieving the substitute words from a local store, in which the examiner 
interprets the memory (12) to be an equivalent to the words from a local store as 
recited in claim 12. By incorporating substituted, recorded, and stored local 
words, one of ordinary skill in the art would provide a learning aid to enhance a 
child's interest in reading. 

Therefore, it would have been obvious to one of ordinary skill in the art at 
the time of the invention was made to modify Reber to further include the 
associated action is speech and the speech incorporates both text from the book 
and substitute words, retrieving the substitute words from a local store, 
soliciting words from a child, recording the child's words, and using the recorded 
word as the substitute words, and the substitute words customize the book text 
to a particular child or locale as taught by La Tour and Linebarger to enhance a 
child's interest in reading. 

24. Claims 7-8, 15-20 are rejected under 35 U.S.C. 103(a) as being unpatentable over Reber 
^261 in view of Rhoads x 436, La Tour ^070 and Linebarger ^485 as applied to claims 2 and 11 
above, and further in view of Rhoads x 214. 

25. Reber in view of Rhoads (^436), La Tour, and Linebarger disclose the claimed invention 
as discussed above except for the following: 

The associated action is playback music as recited in claim 7. 
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The associated action is retrieving artwork from a data store, and printing 
the artwork from the coloring by a child as recited in claim 8. 

Sensing a gesture from the image data, and controlling the action in 
accordance therewith as recited in claim 15. 

One gesture causes a page from the book to be read-aloud again as recited 
in claim 16. 

One gesture controls volume of audio delivered to a speaker as recited in 
claim 17. 

One gesture causes text from the book to be read-aloud at a faster speed 
as recited in claim 18. 

One gesture causes text read-aloud from the book to be read using a 
different voice as recited in claim 19. 

The starting playback of a video at a point corresponding to the page or 
cover as recited in claim 29. 
Rhoads (214) further discloses and teaches the following: 

The associated action is playback music (column 44, lines 17-35) as 
recited in claim 7. 

Sensing a gesture from the image data, and controlling the action in 
accordance therewith (column 17, line 45 - column 20, line 66) as recited in claim 
15. 

One gesture causes a page from the book to be read-aloud again (column 
17, line 45 - column 20, line 66) as recited in claim 16. 
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One gesture controls volume of audio delivered to a speaker (column 17, 
line 45 - column 20, line 66) as recited in claim 17. 

One gesture causes text from the book to be read-aloud at a faster speed 
(column 17, line 45 - column 20, line 66) as recited in claim 18. 

One gesture causes text read-aloud from the book to be read using a 
different voice (column 17, line 45 - column 20, line 66) as recited in claim 19. 

The starting playback of a video at a point corresponding to the page or 
cover (column 59, lines 6-28) as recited in claim 29. By sensing gestures from the 
image data, one of ordinary skill in the art would provide a multimedia enhanced 
interactive experience. 

Therefore, it would have been obvious to one of ordinary skill in the art at 
the time of the invention was made to modify Reber to further include a music 
playback, sensing a gesture from the image data, and controlling the action in 
accordance therewith, one gesture causes a page from the book to be read-aloud 
again, one gesture controls volume of audio delivered to a speaker, one gesture 
causes text from the book to be read-aloud at a faster speed, one gesture causes 
text read-aloud from the book to be read using a different voice, and the starting 
playback of a video at a point corresponding to the page or cover as taught by 
Rhoads (214) to provide a multimedia enhanced interactive experience. 
At the time the invention was made, it would have been an obvious design choice to a 
person of ordinary skill in the art to provide different types of actions because Applicant has not 
disclosed that retrieving artwork from a data store and printing the artwork for coloring by a 
child provides an advantage or solves a stated problem. On of ordinary skill in the art, 
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furthermore, would have expected Applicant's invention to perform equally well with the 
different associated actions as taught by Rhoads (214) because they would provide the same 
function of providing a multimedia enhanced interactive experience. 

26. Claims 21-22 are rejected under 35 U.S.C. 103(a) as being unpatentable over Lo ^943 in 
view of Lemelson ^656 and Rhoads ^436. 

27. Lo discloses the following: 

A book (12), the book having printed pages (figure 3B), each page encoded 
with data, one page being encoded with first data and a further page being 
encoded with second data different than the first data, in which the examiner 
interprets the barcode to be a functionally equivalent to the data (12A and 12B in 
figure 3B), one page of the book presented to the reading station, in which the 
examiner interprets the decoder to be a functional equivalent to the reading 
station, the reading stating having a processor (within decoder), memory (within 
decoder), a speaker, visible light scan data to the processor (figure 4), the 
processor decoding the visible light scan data to decoded the first data, the 
processor accessing stored voice data from the memory in accordance with the 
decoded data and causing the stored voice data to be rendered using the speaker 
(column 2, lines 9-51), the further page of the book, presenting the further page to 
the reading station, in which the examiner interprets the decoder to be a 
functional equivalent to the reading station, the processor of the reading station 
decoding visible light scan data corresponding to the further page to decode the 
second data and accessing stored voice data from the memory in accordance with 
the decoded sound plural bit data and causing the stored voice data to be 
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rendered using the speaker, and the child controls the read-aloud process as 
recited in claim 21. 
Lo does not expressly disclose the following: 

Each page of the book is encoded with different steganographic plural bit 
data and the different stored voice data for other pages as recited in claim 21. 

One of the pages being steganographically encoded with plural bit data 
and another of the pages being steganographically encoded with different plural 
bit data, and the steganographic encoding not being apparent to human observers 
of the pages, but can be decoded from image data produced by visible light 
scanning of the pages as recited in claim 22. 
Lemelson teaches the following: 

Accessing different stored voice for the different data as recited in claim 
21. By having different stored voice for different data, one of ordinary skill in the 
art would provide a child or handicapped person to derive intelligible 
information from a book. 
Rhoads teaches the following: 

Providing embedded information having different steganographic encoded 
plural bit data not being apparent to human observers of the different embedded 
information, but can be decoded from image data produced by visible light 
scanning of the embedded information as recited in claim 22. By having 
steganographic encoded plural bit data not being apparent to human observers of 
different embedded information, one of ordinary skill in the art would provide a 
multimedia enhanced interactive experience. 



Application/Control Number: 09/709,255 Page 18 

Art Unit: 3714 

Therefore, it would have been obvious to one of ordinary skill in the art at 
the time of the invention was made to modify Lo to include each page of the book 
is encoded with different steganographic plural bit data and the different stored 
voice data for other pages as taught by Lemelson and one of the pages being 
steganographically encoded with plural bit data and another of the pages being 
steganographically encoded with different plural bit data, and the steganographic 
encoding not being apparent to human observers of the pages, but can be 
decoded from image data produced by visible light scanning of the pages as 
taught by Rhoads to provide a multimedia enhanced interactive experience. 

Conclusion 

28. The prior art made of record and not relied upon is considered pertinent to applicant's 
disclosure. 

Rhoads ^834, Redford ^507, and Cooperman T>04 disclose different types of watermark 
encoding and decoding data. 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Alex P. Rada whose telephone number is 703-308-7135. The 
examiner can normally be reached on Monday - Friday, 08:00-16:00. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's Primary, 
Jessica Harrison can be reached on 703-308-2217. The fax phone number for the organization 
where this application or proceeding is assigned is 703-872-9306. 
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